Skip to content

Conversation

@phernandez
Copy link
Member

Summary

  • add fast feedback loop targets (testmon, test-smoke, fast-check, doctor) in justfile
  • add basic-memory doctor command for a clean, self-checking loop
  • add MCP smoke + tool-contract tests to validate tool signatures
  • wire up pytest-testmon, smoke marker, and ignore .testmondata*
  • refresh docs for the code/test/verify loop and keep CLAUDE.md as a symlink to AGENTS.md
  • include minor formatting fixes in existing tests

Why

  • shorten the local verification loop for agent-driven changes
  • catch tool-contract drift and basic integration issues earlier
  • make it easy to run a clean, reproducible “does this work on my machine?” check

How

  • added a doctor CLI entry point that runs with a temporary HOME/config
  • introduced focused MCP smoke/contract tests to validate tool names and args
  • standardized the fast path in justfile and documented the loop

Testing

  • just fast-check
  • just test-smoke
  • uv run pytest tests/mcp/test_tool_contracts.py
  • just doctor

@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2026

Claude finished @phernandez's task in 1m 17s —— View job


Reviewing PR: Fast Feedback Loop Tooling

Todo List

  • Read and understand changed files
  • Review code quality & standards
  • Review testing & documentation
  • Review Basic Memory architecture compliance
  • Check security & performance
  • Check compatibility
  • Post comprehensive review feedback

Review Summary

This PR successfully adds fast feedback loop tooling to improve the development workflow. The implementation is solid with good test coverage and documentation. I've identified a few items that need attention:

✅ Code Quality & Standards

  • Follows Basic Memory's coding conventions in CLAUDE.md
  • Python 3.12+ type annotations and async patterns
  • SQLAlchemy 2.0 best practices
  • FastAPI and Typer conventions followed
  • 100-character line length limit maintained
  • No commented-out code blocks

Observations:

  • Code follows project conventions well
  • Type hints are properly used throughout
  • Async patterns correctly implemented

⚠️ Testing & Documentation

  • Unit tests for new functions/methods (test_tool_contracts.py)
  • Integration tests for new MCP tools (test_smoke_integration.py)
  • Test coverage for edge cases
  • 100% test coverage maintained - doctor.py is missing test coverage
  • Documentation updated (README, AGENTS.md)
  • CLAUDE.md updated (now a symlink to AGENTS.md)

Issue: The new doctor.py command (src/basic_memory/cli/commands/doctor.py:1-154) lacks dedicated unit tests. While it's exercised via the just doctor integration test, CLAUDE.md states: "Coverage must stay at 100%: Write tests for new code." The CLI omit pattern in pyproject.toml:143 may cover this, but please verify coverage doesn't drop.

Fix test coverage →

✅ Basic Memory Architecture

  • MCP tools follow atomic, composable design
  • Database changes include Alembic migrations (N/A for this PR)
  • Preserves local-first architecture principles
  • Knowledge graph operations maintain consistency
  • Markdown file handling preserves integrity
  • AI-human collaboration patterns followed

Observations:

  • The doctor command properly uses temporary directories and isolated config
  • MCP smoke tests validate end-to-end tool contracts
  • Follows the async client pattern correctly (get_client() context manager)

✅ Security & Performance

  • No hardcoded secrets or credentials
  • Input validation for MCP tools
  • Proper error handling and logging
  • Performance considerations addressed
  • No sensitive data in logs or commits

Observations:

  • doctor.py properly handles cleanup in finally block (lines 128-130)
  • Temporary directories are used securely
  • Error handling covers ToolError, ValueError, and general exceptions

⚠️ Compatibility

  • File path comparisons must be windows compatible
  • Avoid using emojis and unicode characters in console and log output

Issue: The justfile:8 contains an emoji: @echo "💡 Remember to activate...". While this is in the justfile (not code), CLAUDE.md explicitly states to avoid emojis in console output unless the user explicitly requests it.

Fix emoji in justfile →

📋 Additional Observations

Positive Highlights:

  1. Excellent documentation updates - The fast path workflow is clearly documented in AGENTS.md (lines 47-54)
  2. Smart symlink approach - Converting CLAUDE.md to a symlink pointing to AGENTS.md (lines 1-408) reduces duplication
  3. Comprehensive test contracts - test_tool_contracts.py will catch MCP tool signature drift early
  4. testmon integration - Using pytest-testmon for impacted test detection is a smart optimization
  5. Proper .gitignore entry - .testmondata* added at line 4

Minor Formatting Fixes:

  • test_entity_parser_error_handling.py has proper spacing improvements (no issues)
  • test_search_repository.py has cleaned up imports (removed unused from datetime import timezone, datetime)

🎯 Verdict

This PR adds valuable tooling for the development workflow. The implementation is solid, but needs attention on:

  1. Test coverage for doctor.py (high priority if not covered by CLI omit pattern)
  2. Emoji removal from justfile (low priority, style compliance)

The core functionality is excellent:

  • just fast-check - Smart fast path for local development
  • just testmon - Runs only impacted tests
  • just test-smoke - Quick MCP end-to-end validation
  • just doctor - Self-checking file ↔ DB loop
  • Tool contract tests prevent signature drift

Recommend addressing test coverage verification before merging. The emoji fix is optional but recommended for consistency.


@phernandez phernandez changed the title Add fast feedback loop tooling dev: Add fast feedback loop tooling Feb 2, 2026
@phernandez phernandez changed the title dev: Add fast feedback loop tooling chore: Add fast feedback loop tooling Feb 2, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

Claude finished @phernandez's task in 3m 41s —— View job


PR Review: Fast Feedback Loop Tooling

This PR adds valuable fast feedback loop tooling to improve the development workflow. I've reviewed it against the Basic Memory team checklist.

✅ Code Quality & Standards

Passing:

  • Follows Basic Memory's coding conventions in CLAUDE.md/AGENTS.md
  • Python 3.12+ type annotations and async patterns used correctly
  • SQLAlchemy 2.0 best practices (N/A - no DB changes)
  • FastAPI and Typer conventions followed correctly
  • 100-character line length limit maintained
  • No commented-out code blocks

Observations:

  • doctor.py properly uses async patterns with get_client() context manager
  • Type hints are comprehensive throughout
  • Code follows repository async client pattern (SPEC-16 compliant)
  • CLI routing flags (--local/--cloud) properly implemented

✅ Testing & Documentation

Passing:

  • Unit tests for MCP tool contracts (test_tool_contracts.py)
  • Integration smoke test (test_smoke_integration.py)
  • Test coverage for edge cases
  • 100% test coverage maintained - doctor.py covered by CLI omit pattern (pyproject.toml:143)
  • Documentation updated (README.md lines 494-533, AGENTS.md lines 47-54)
  • CLAUDE.md updated (now symlink to AGENTS.md)

Observations:

  • Smoke test validates full write → read → search → build_context flow
  • Tool contract test will catch signature drift early (excellent!)
  • .testmondata* properly added to .gitignore:4
  • README clearly documents fast path workflow
  • Minor formatting fixes in test files (spacing improvements, removed unused imports)

✅ Basic Memory Architecture

Passing:

  • MCP tools follow atomic, composable design
  • Database changes include Alembic migrations (N/A for this PR)
  • Preserves local-first architecture principles
  • Knowledge graph operations maintain consistency
  • Markdown file handling preserves integrity
  • AI-human collaboration patterns followed

Observations:

  • doctor command creates isolated temp project with temp HOME/config (lines 40, 172-178 in justfile)
  • Proper cleanup in finally block (doctor.py:128-130)
  • Uses EntityParser and MarkdownProcessor correctly
  • Validates full file → DB → search → status loop

✅ Security & Performance

Passing:

  • No hardcoded secrets or credentials
  • Input validation for MCP tools (contract tests ensure signature stability)
  • Proper error handling and logging
  • Performance considerations addressed (testmon for impacted tests only)
  • No sensitive data in logs or commits

Observations:

  • Error handling covers ToolError, ValueError, and general Exception
  • Temporary directories cleaned up properly
  • Fast path optimized with pytest-testmon

⚠️ Compatibility

Needs Attention:

  • File path comparisons are Windows compatible (Path objects used)
  • Emojis in console output - justfile contains multiple emojis

Issue: The justfile has many emojis in echo statements (lines 8, 61, 86, 96, 132, 199, 206, 209, etc.). While the custom instructions mention "Avoid using emojis and unicode characters in console and log output", these are in build scripts (not Python code). The previous review flagged line 8 specifically.

The justfile emojis appear in:

  • Installation reminder (💡 line 8)
  • Warnings (⚠️ lines 61, 132)
  • Success messages (✅ lines 86, 96, 253)
  • Release workflow messages (🚀📋❌🔍📝🏷️📤🔗📦 throughout release targets)

Recommendation: This is a judgment call. The emoji guideline seems to target Python console/log output (which doctor.py correctly avoids). The justfile emojis are developer-facing build output, not user-facing application output. However, for strict compliance with the compatibility checklist, they should be removed or replaced with plain text markers.

Fix justfile emojis →

📋 Summary

Strengths

  1. Excellent fast path workflow - just fast-check provides quick local verification
  2. Smart test optimization - pytest-testmon for impacted tests only
  3. Tool contract validation - Will catch MCP signature drift early
  4. Comprehensive smoke test - Full end-to-end MCP flow validated
  5. Clean isolation - doctor runs with temp HOME/config (won't touch user settings)
  6. Good documentation - README and AGENTS.md clearly document the workflow
  7. CLAUDE.md symlink - Reduces duplication while maintaining compatibility

Implementation Quality

  • Clean async patterns throughout
  • Proper error handling and cleanup
  • Type hints are comprehensive
  • Follows project conventions

Areas for Consideration

  • Emojis in justfile - Technically violates compatibility checklist, though these are developer-facing build outputs

🎯 Verdict

Recommend: Approve with optional emoji cleanup

The core functionality is excellent and ready to merge. The emoji issue is minor and debatable (build script output vs application output). If strict compliance is desired, remove them. Otherwise, the PR successfully delivers:

  • ✅ Fast local verification loop
  • ✅ MCP tool contract tests
  • ✅ Smoke tests for quick validation
  • ✅ Doctor command for consistency checks
  • ✅ Good documentation

The tooling will significantly improve the development workflow, especially for agent-driven changes.


@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2026

Claude finished @phernandez's task in 2m 7s —— View job


PR Review: Fast Feedback Loop Tooling

This PR adds valuable fast feedback loop tooling to improve the development workflow. I've reviewed it against the Basic Memory team checklist.


Code Quality & Standards

Status: ✅ PASSING

  • Follows Basic Memory's coding conventions in CLAUDE.md/AGENTS.md
  • Python 3.12+ type annotations and async patterns used correctly
  • SQLAlchemy 2.0 best practices (N/A - no DB changes in this PR)
  • FastAPI and Typer conventions followed correctly
  • 100-character line length limit maintained
  • No commented-out code blocks

Observations:

  • doctor.py (lines 1-154) properly uses async patterns with get_client() context manager
  • Type hints are comprehensive throughout all new code
  • Code follows repository async client pattern (SPEC-16 compliant)
  • CLI routing flags (--local/--cloud) properly implemented in doctor.py:136-141
  • Imports follow proper ordering: stdlib → third-party → local

Testing & Documentation

Status: ✅ PASSING

  • Unit tests for MCP tool contracts (tests/mcp/test_tool_contracts.py)
  • Integration smoke test (test-int/mcp/test_smoke_integration.py)
  • Test coverage for edge cases
  • 100% test coverage maintained - doctor.py covered by CLI omit pattern (pyproject.toml:143)
  • Documentation updated (README.md lines 494-533, AGENTS.md lines 47-54)
  • CLAUDE.md updated (now symlink to AGENTS.md)

Observations:

  • Excellent smoke test (test_smoke_integration.py:1-56): Validates full write → read → search → build_context flow end-to-end
  • Smart tool contract test (test_tool_contracts.py:1-89): Will catch MCP signature drift early before it breaks clients
    • Uses inspect.signature() to validate all 17 MCP tools maintain expected signatures
    • Particularly valuable for preventing breaking changes to tool APIs
  • Coverage properly configured: The doctor.py CLI command is covered by the */cli/** omit pattern (pyproject.toml:143), which is appropriate since "core logic is covered via API/MCP/service tests"
  • .testmondata* properly added to .gitignore:4 - prevents pytest-testmon cache pollution
  • Minor test improvements:
    • test_entity_parser_error_handling.py: Better spacing (no functional change)
    • test_search_repository.py: Removed unused imports from datetime import timezone, datetime
  • README documentation (lines 494-533): Clearly documents fast path workflow and new commands
  • AGENTS.md documentation (lines 47-54): Documents Code/Test/Verify Loop with clear workflow steps
  • CLAUDE.md → AGENTS.md symlink: Smart approach to reduce duplication while maintaining compatibility

Test markers properly used:

  • @pytest.mark.smoke for smoke tests
  • @pytest.mark.asyncio for async test support

Basic Memory Architecture

Status: ✅ PASSING

  • MCP tools follow atomic, composable design
  • Database changes include Alembic migrations (N/A for this PR)
  • Preserves local-first architecture principles
  • Knowledge graph operations maintain consistency
  • Markdown file handling preserves integrity
  • AI-human collaboration patterns followed

Observations:

  • doctor command design (doctor.py:31-133): Excellent isolated testing approach
    • Creates isolated temp project with temp HOME/config (justfile:172-178)
    • Tests full bidirectional flow:
      1. DB → File: Create entity via API, verify file written (lines 60-80)
      2. File → DB: Write markdown file manually, sync, verify in search (lines 82-119)
      3. Status verification: Ensures clean state after sync (lines 121-126)
    • Proper cleanup in finally block (lines 128-130)
    • Uses EntityParser and MarkdownProcessor correctly
    • Validates full file → DB → search → status loop
  • MCP smoke test validates end-to-end tool contracts without mocking
  • Tool contract test ensures API stability for external clients

Security & Performance

Status: ✅ PASSING

  • No hardcoded secrets or credentials
  • Input validation for MCP tools (contract tests ensure signature stability)
  • Proper error handling and logging
  • Performance considerations addressed (testmon for impacted tests only)
  • No sensitive data in logs or commits

Observations:

  • Error handling (doctor.py:147-153): Covers ToolError, ValueError, and general Exception
    • Line 153 properly marked with # pragma: no cover for the general exception handler
  • Temporary directories cleaned up properly via context manager (line 40) and finally block (lines 128-130)
  • Fast path optimized with pytest-testmon for impacted tests only
  • No security concerns in new code

Compatibility

Status: ⚠️ MINOR ISSUE

  • File path comparisons are Windows compatible (Path objects used throughout)
  • Emojis in console output - justfile contains emojis in echo statements

Issue Details:

The justfile contains emojis in multiple echo statements. The custom instructions state: "Avoid using emojis and unicode characters in console and log output." Examples:

  • Line 8: 💡 Remember to activate...
  • Line 61: ⚠️ No timeout command found...
  • Lines 86, 96, 253: success markers
  • Throughout release targets: 🚀📋❌🔍📝🏷️📤🔗📦

Context and Recommendation:

This is a judgment call. The emoji guideline appears to target Python application console/log output (which doctor.py correctly avoids). The justfile emojis are in developer-facing build script output, not user-facing application output.

Options:

  1. Accept as-is: The emojis are in build tooling, not application code
  2. Remove for strict compliance: Replace with plain text equivalents

Given that:

  • Previous reviews flagged this as a compatibility concern
  • The custom instructions explicitly state to avoid emojis in console output
  • The justfile is indeed console output (developer-facing)

Recommendation: For strict compliance with the compatibility checklist, remove or replace with plain text. However, this is low priority and doesn't affect functionality.

Fix justfile emojis →


Summary

Strengths ✨

  1. Excellent fast path workflow - just fast-check provides quick local verification loop
  2. Smart test optimization - pytest-testmon runs only impacted tests (significant time savings)
  3. Tool contract validation - Will catch MCP signature drift early before breaking external clients
  4. Comprehensive smoke test - Full end-to-end MCP flow validated (write → read → search → build_context)
  5. Clean isolation - doctor runs with temp HOME/config (won't touch user settings)
  6. Good documentation - README and AGENTS.md clearly document the workflow
  7. CLAUDE.md symlink - Reduces duplication while maintaining compatibility
  8. Proper error handling - Covers multiple exception types with appropriate exit codes
  9. Type safety - Comprehensive type hints throughout

Implementation Quality 📊

  • Clean async patterns throughout doctor.py using get_client() context manager
  • Proper resource cleanup via context managers and finally blocks
  • Type hints are comprehensive in all new code
  • Follows project conventions consistently
  • pytest markers properly used (@pytest.mark.smoke, @pytest.mark.asyncio)

Areas for Consideration 🤔

  • Emojis in justfile - Technically violates compatibility checklist for console output
    • Low priority, debatable (build scripts vs application code)
    • See fix link above if strict compliance desired

New Capabilities 🚀

This PR delivers:

  • just fast-check - Fast local verification loop (lint/format/typecheck + impacted tests + smoke)
  • just testmon - Run only tests impacted by recent changes
  • just test-smoke - Fast MCP end-to-end validation
  • just doctor / basic-memory doctor - Local file ↔ DB consistency checks
  • MCP tool contract tests - Prevents signature drift
  • MCP smoke tests - Quick integration validation

Verdict

✅ RECOMMEND: APPROVE (with optional emoji cleanup)

The core functionality is excellent and ready to merge. The emoji issue is minor and debatable (build script output vs application output). If strict compliance is desired, use the fix link above to remove them. Otherwise, the PR successfully delivers valuable tooling that will significantly improve the development workflow, especially for agent-driven changes.

Final Checklist:

  • ✅ Code quality and standards
  • ✅ Testing and documentation
  • ✅ Basic Memory architecture compliance
  • ✅ Security and performance
  • ⚠️ Compatibility (emoji style issue, optional fix)

The tooling will provide excellent value for rapid iteration and catching issues early.


@phernandez phernandez merged commit 8072449 into main Feb 2, 2026
28 checks passed
@phernandez phernandez deleted the close-loop-tooling branch February 2, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants